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;_( We study cooperation in multi terminal source coding models involving successive refinement. 
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^) Specifically, we study the case of a single encoder and two decoders, where the encoder provides a common 

^»o description to both the decoders and a private description to only one of the decoders. The decoders 

cooperate via cribbing, i.e., the decoder with access only to the common description is allowed to observe, 

^H in addition, a deterministic function of the reconstruction symbols produced by the other. We characterize 

hH 
^ the fundamental performance limits in the respective settings of non-causal, strictly-causal and causal 

O 
I— —I cribbing. We use a new coding scheme, referred to as Forward Encoding and Block Markov Decoding, 

T— I which is a variant of one recently used by Cuff and Zhao for coordination via implicit communication. 
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l/~j Finally, we use the insight gained to introduce and solve some dual channel coding scenarios involving 
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00 Multiple Access Channels with cribbing. 
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I. Introduction 

Cooperation can dramatically boost the performance of a network. The literature abounds with models 
for cooperation, when communication between nodes of a network is over a noisy channel. In multiple 
access channels, the setting of cribbing was introduced by Willems and Van der Muelen in [1], where 
one encoder obtains the channel input symbols of the other encoder (referred to as "crib") and uses it 
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for coding over a multiple access channel (MAC). This was further generalized to deterministic function 
cribbing (where an encoder obtains a deterministic function of the channel input symbols of another 
encoder) and to cribbing with actions (where one encoder can control the quality and availability of the 
"crib" by taking cost constrained actions) by Permuter and Asnani in [2]. Cooperation can also be modeled 
as information exchange among the transmitters and receivers via rate limited links, generally referred to 
as conferencing in the literature. Such a model was introduced in the context of the MAC by Willems 
in [3], and subsequently studied by by Bross, Lapidoth and Wigger [4], Wiese et al. [5], Simeone et al. 
[6], and Marie, Yates and Kramer [7]. Cooperation has also been modeled via conferencing/cribbing in 
cognitive interference channels, such as the settings in Bross, Steinberg and Tinguely [8] and Prabhakaran 
and Vishwanath [9]-[10]. We refer to Ng and Goldsmith [11] for a survey of various cooperation strategies 
and their fundamental limits in wireless networks. 
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Fig. 1. Cascade source coding setup. 
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Fig. 2. Triangular source coding setup. 



3 
In multi terminal source coding, cooperation is generally modeled as a rate limited link such as in the 
cascade source coding setting of Yamamoto [12], Cuff, Su and El Gamal [13], Permuter and Weissman 
[14], Chia, Permuter and Weissman [15], as well as the triangular source coding problems of Yamamoto 
[16], Chia, Permuter and Weissman [15]. In cascade source coding (Fig. 1), Decoder 1 sends a description 
(T12) to Decoder 2, which does not receive any direct description from the encoder, while in triangular 
source coding (Fig. 2), Decoder 1 provides a description (T12) to Decoder 2 in addition to the direct 
description (T2) from the encoder. 

The contribution of this paper is to introduce new models of cooperation in multi terminal source coding, 
inspired by the cribbing of Willems and Van der Muelen [1] and by the implicit communication model 
of Cuff and Zhao [17]. Specifically, we consider cooperation between decoders in a successive refinement 
setting (introduced in Equitz and Cover [18]). In successive refinement, a single encoder describes a 
common rate to both the decoders and a private rate to only one of the decoders. We generalize this model 
to accommodate cooperation among the decoders as follows : 
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Fig. 3. Successive refinement, with decoders cooperating via conferencing. 

1) Cooperation via Conferencing : One such cooperation model considered is that shown in Fig. 3, where 
the encoder provides a common description (Tq) to both the decoders and a refined description (Ti) 
to Decoder 1, Decoder 1 cooperates with Decoder 2 by providing an additional description (T12) 
which is the function of its own private description (Ti), as well as the common description (Tq). 
This setting is inspired by the conferencing problem in channel coding described earlier. The region 
of achievable rates and distortions for this problem is given by, 

R^ + Ri > I{X-Xi,X2) (1) 

R^ + Ri2 > I{X;X2), (2) 
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for some joint probability distribution P^ x x ^^^^ that E[(ij(Xj,Xj)] < Di, for i = 1,2, where 
di refers to the distortion function and Di are the distortion constraints, as is formally explained 
in Section II. The direct part of this characterization, namely that this region is achievable, follows 
standard arguments that generalize those used in the original successive refinement problem [18] (cf. 
Appendix A). 
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Fig. 4. Successive reiinement, witli decoders cooperating via cribbing. d = n, d — i~l and d = i respectively correspond to 
non-causal, strictly-causal and causal cribbing. 



2) Cooperation via Cribbing : The main setting analyzed in this paper is shown in Fig. 4. A single 
encoder describes a common message To to both decoders and a refined message Ti to only Decoder 
1. Instead of cooperating via a rate limited link, as in Fig. 3, Decoder 2 "cribs" (in the spirit of 
Willems and Van der Muelen [1]) a deterministic function g of the reconstruction symbols of Decoder 
1, non-causally, strictly-causally, or causally. Note a trivial g function corresponds to the original 
successive refinement setting characterized in Equitz and Cover [18]. The goal is to find the optimal 
encoding and decoding strategy and to characterize the optimal encoding rate region which is defined 
as the set of achievable rate tuples {Rq, Ri) such that the distortion constraints are satisfied at both 
the decoders. Cuff and Zhao [17], considered the problem of characterizing the coordination region 
(non-causal, strictly causal and causal coordination) in our setting of Fig. 4, for a specific function, g, 
such that g{Xi) = Xi and for a specific rate tuple {Rq, Ri) = (0, oo), that is Decoder 1 has access 
to the source sequence X^ while Decoder 2 uses the reconstruction symbols of Decoder 1 (non- 
causally, strictly-causally or causally) to estimate the source. We use a new source coding scheme 
which we refer to as Forward Encoding and Block Markov Decoding, and show that it achieves the 
optimal rate region for strictly causal and causal cribbing. It draws on the achievability ideas (for 
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causal coordination) introduced in Cuff and Zhao [17]. This scheme operates in blocks, where in 
the current block, the encoder encodes for the source sequence of the future block, (hence the name 
Forward Encoding) and the decoders rely on the decodability in the previous block to decode in the 
current block (hence the name Block Markov Decoding). More details about this scheme are deferred 
to Section III. 

The general motivation for our work is an attempt to understand fundamental limits in source coding 
scenarios involving the availability of side information in the form of a lossily compressed version of the 
source. This is a departure from the standard and well studied models where side information is merely a 
correlated "noisy version" of the source, and is challenging because the effective 'channel' from source to 
side information is now induced by a compression scheme. Thus, rather than dictated by nature, the side 
information is now another degree of freedom in the design. There is no shortage of practical scenarios 
that motivate our models. 

One such scenario may arise in the context of video coding, as considered by Aaron, Varodayan and 
Girod in ([19]). Consider two consecutive frames in a video file, denoted by Frame 1 and Frame 2, 
respectively. The video encoder starts by encoding Frame 1, and then it encodes the difference between 
Frame 1 and Frame 2. Decoder 1 represents decoding of Frame 1, while Decoder 2 uses the knowledge 
of decoded Frame 1 (via cribbing) to estimate the next frame. Frame 2. 

Our problem setting is equally natural for capturing noncooperation as it is for capturing cooperation, 
by requiring the relevant distortions to be bounded from below rather than above (which, in turn, can be 
converted to our standard form of an upper bound on the distortion by changing the sign of the distortion 
criterion). For instance. Decoder 1 can represent an end-user with refined information (common and private 
rate) about a secret document, the source in our problem, while Decoder 2 has a crude information about the 
document (via the common rate). Decoder 1 is required to publicly announce an approximate version of the 
document, but due to privacy issues would like to remain somewhat cryptic about the source (as measured 
in terms of distortion with respect to the source) while also helping (via conferencing or cribbing) Decoder 
2 to better estimate the source. For example. Decoder 1 can represent a Government agency required by 
law to publicly reveal features of the data, while on the other hand there are agents who make use of this 
publicly announced information, along with crude information about the source that they too, not only the 



government, are allowed to access, to decipher or get a good estimate of the classified information (the 
source). 

The contribution of this paper is two-fold. First, we introduce new models of decoder cooperation in 
source coding problems such as successive refinement, where decoders cooperate via cribbing, and we 
characterize the fundamental limits on performance for these problems using new classes of schemes for 
the achievability part. Second, we leverage the insights gained from these problems to introduce and solve 
a new class of channel coding scenarios that are dual to the source coding ones. Specifically, we consider 
the MAC with cribbing and a common message, where there are two encoders who want to communicate 
messages over the MAC, one has access to its own private message, there is a common message between 
the two encoders, and the encoders cooperate via cribbing (non-causally, strictly causally or causally). 

The paper is organized as follows. Section II gives a formal description of the problem and the main 
results. Section III presents achievability and converses, with non-causal, causal and strictly-causal cribbing. 
Some special cases of our setting and numerical examples, are studied in Section IV. Channel coding duals 
are considered in Section V. Finally, the paper is concluded in Section VI. 

II. Problem Definitions and Main Results 

We begin by explaining the notation to be used throughout this paper. Let upper case, lower case, and 
calligraphic letters denote, respectively, random variables, specific or deterministic values which random 
variables may assume, and their alphabets. For two jointly distributed random variables, X and Y, let 
Px, PxY and Px\y respectively denote the marginal of X, joint distribution of {X,Y) and conditional 
distribution of X given Y . X^^ is a shorthand for the n — m + I tuple {Xm, Xm+i, • • • , Xn-i,Xn}- We 
impose the assumption of finiteness of cardinality on all alphabets, unless otherwise indicated. 

In this section we formally define the problem considered in this paper (cf. Fig. 4). The source sequence 
Xi (^ X,i = 1,2,... is drawn i.i.d. r^ Px- Let A'l and X2 denote the reconstruction alphabets, and di : 
X X Xi ^ [0,00), for i = 1,2 denote single letter distortion measures. Distortion between sequences is 
defined in the usual way, 

1 " 
cii(x",x") = -^(ii(xj,Xjj), fori = 1,2. (3) 

""7=1 



Definition 1. A (2"'^° , 2"^^ , n) rate-distortion code consists of the following, 

1) Encoder, /o,„ : A-" ^ {1, ..., 2"««}. h,n : ^" ^ {1, -, 2"^^}. 

2) Decoder 1, 5i,„ : {1, ..., 2"-^«} x {1, ..., 2"-^i} ^ i'[\ 

3) Decoder 2 (depending on d in Fig. 4, the decoder mapping changes as below), 

^2,1 : {1, ...,2"^"} X Xi -^ X2 non-causal cribbing, d = n (4) 

^l^j : {1, ..., 2"-^"} X X[^^ -> X2 strictly -causal cribbing, d = i - I, (5) 

52,1 : {1, •••,2"^°} X A'{ -> ^"2 causal cribbing, d = i (6) 

V i = 1, ...,n. 

Definition 2. A rate-distortion tuple {Ro,Ri,Di,D2) is said to be achievable if \/ e > 0, 3 n and 
^2««o ^ 2"^i ^ 7T,j rate-distortion code such that the expected distortion for decoders are bounded as. 



di{Xf',Xf' 



<Di + e, i = l,2. (7) 



Definition 3. The rate-distortion region TZ{Di,D2) is defined as the closure of the set to all achievable 
rate-distortion tuples {Rq,Ri,Di,D2). 

Our main results for this setting are presented in the Table I. Note that in all the rate regions in the 
table, we use the notation {a}+ for max(a,0), and we omit the distortion condition E[di{Xi, Xi] < Di, 
i = 1 , 2 for the sake of brevity. These results will be derived later in Section III. As another contribution, 
in Section V, we establish duality between the problem of successive refinement with cribbing decoders 
and communication over multiple access channels with cribbing encoders and a common message. We 
establish a complete duality between the settings (in a sense that is detailed in Section V) and rate regions 
of one can be obtained from those of the other by listed transformations. 

Lemma 1 (Equivalence to Cascade Source Coding with Cribbing Decoders). The setup in Fig. 4 is 
equivalent to a cascade source coding setup with cribbing decoders as in Fig. 5 in the following way : 
fix a distortion pair {Di,D2) and let TZ{Di,D2) denote the rate region for the problem of successive 
refinement with cribbing with achievable rate pairs {Rq,Ri). Let R{Di,D2) denote the closure of 
rate pairs, {Rq,Rq + i?i) and TZcascadeiDi, D2) denote the rate region for the problem of cascade 
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Deterministic Function 
Cribbing 


Non-Causal 

{d = n) 


(Theorem 1) 

Ro + Ri >I{X-Xi,X2) 
Ro>{IiX;XuX2)-HiXi)}+ 

(p.m.f.):P{X,Xi,X2) 


(Theorem 2) 

Rq + Ri>I{X;Xi,X2) 
Ro>{I{X;Zi,X2)-H{Zi)}+ 

(p.m.f.):P{X,X,,X2)l^z.=^^j,^)} 


Strictly-Causal 

(d = i-l) 


(Theorem 3) 

Ro > {liX; Xi, X2) - i7(Xi|X2)}+ 
(p.m.f.):P{X,Xi,X2) 


(Theorem 4) 

Ro + Ri>IiX;Xi,X2) 
Ro>{IiX;Zi,X2)-HiZi\X2)V 

(p.m.f.):PiX,Xi,X2)l{z,=g(x,)} 


Causal 


(Theorem 5) 

Ro + Ri>I{X;Xi,U) 

Ro>{I{X;XuU)-H{Xi\U)}+ 

(p.m.f.):P{X,X,,U)l^ji^^^^^)^ 

\U\ < \X\\Xi\+4. 


(Theorem 6) 

Ro + Ri>I{X;Xi,U) 

Ro>{I{X;Z,,U)-H{Zi\U)}+ 

(p.m.f.) : P{X,Xi,U)l^z,=g{x,),x,=f{x„u)} 

\U\ < \X\\Xl\+4: 



TABLE I 

Main Results of the Paper 



source coding with cribbing (closure of achievable rate pairs {Ri2,Ri))- We then have the equivalence, 

n{Di,D2)=ncascade{Dl,D2). 

Proof: Proof is similar to the proof of Theorem 3 in Vasudevan, Tian and Diggavi [20] . We state it 
in Appendix B for quick reference. ■ 

We use certain standard techniques such as Typical Average Lemma, Covering Lemma and Packing 
Lemma which are stated and established in [21]. Herein, we state them for the sake of quick reference. 
For typical sets we use the definition as in chapter 2 of [21]. Henceforth, we omit the alphabets from the 
notation of typical set when it is clear from context, e.g. 7^"(X, X2) is denoted by 7^". 

Lemma 2 (Typical Average Lemma, Chapter 2, [21]). Let x"- G T". Then for any nonnegative function 
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Fig. 5. Cascade source coding withi cribbing decoders, d = n, d = i — 1 and d = i respectively correspond to non-causal, 
strictly-causal and causal cribbing. 



g{x) on X, 



1 "^ 

(1 - e)Eb(X)] < -Y9{x.) < (1 + 6)Eb(X)]. 



(8) 



1=1 



Lemma 3 (Covering Lemma, Chapter 3, [21]). Let {U,X,X) r^ p{u,x,x). Let {U"',X"-) ~ p(n",x") be 
a pair of arbitrarily distributed random sequences such that P{(f^",X") G T"} — t- 1 as' n — )■ 00 awJ let 
X"'{m),m G A where \A\ > 2"^, Z?e random sequences, conditionally independent of each other and 
of X^ given [/"", each distributed according to ]Xi=iPx\Tjixi\ui). Then, there exists 8{e) — )■ such that 
P{{U'', X", l"(m)) (^T^ymeA}^Oasn^oo,ifR> I{X; X\U) + 6{e). 

Lemma 4 (Packing Lemma, Chapter 3, [21]). Let {U,X,Y) ~ p{u,x,y). Let ([/",y") ~ p{u'\y'') be 
a pair of arbitrarily distributed random sequences (not necessarily according to YYl^iPuxi^i'Vi))- ^^^ 
X"'{m),m G A, where \A\ < 2"^, be random sequences, each distributed according to YVi=iPx\u^^^\^^^' 
Assume that X'^{m),m G A, is pairwise conditionally independent of Y^ given [/", but is arbitrarily 
dependent on other X'^{ra) sequences. Then, there exists (5(e) — )■ such that P{U^,X'^,Y^{m)) G 
T^y me A} ^0 as n^ 00 , if R< I{X; Y\U) + (5(e). 



III. Successive Refinement with Cribbing Decoders 

In this section we analyze the main settings considered in this paper and derive rate regions. In the 
various subsections to follow we will respectively study the problem of successive refinement with non- 
causal, strictly causal and causal cribbing. For clarity, in each subsection, we will first study the setting of 
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"perfect" cribbing where Zi j = g{Xi^,i) = Xi j and then generaUze it to cribbing with any deterministic 
function g. 

A. Non-causal Cribbing 
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Fig. 6. Successive refinement, with decoders cooperating via (perfect) non-causal cribbing. 



1) Perfect Cribbing: 

Theorem 1. The rate region TZ{Di,D2) for the setting in Fig. 6 with perfect (non-causal) cribbing is 
given as the closure of the set of all the rate tuples {Rq,Ri) such that, 

Ro + Ri > I{X-Xi,X2) (9) 

Ro > {I{X;Xi,X2)-H{Xi)}+, (10) 

for some joint probability distribution P^ j^ ^ such that E[di{X, Xi)] < Di, for i = 1,2. 

Proof: 

Achievability : 

"Double Binning" scheme 

Before delving into the details, we first provide a high level understanding of the achievability scheme. 
Consider the simplified setup where Rq = 0, that is only Decoder 1 has access to the description of the 
source, and Decoder 2 gets the reconstruction symbols of Decoder 1 ("crib"). The intuition is to reveal 
a lossy description of source to the Decoder 2 through the "crib". So we first generate 2'^^^^'''^^'i X2 
codewords, and index them as 2"^^-^'^^^ bins. In each bin, we generate a superimposed codebook of 
2ni{x-X\X2) xn codewords. Thus total rate of Ri = I{X; X2) + I{X; X1IX2) = I{X; Xi, X2) is needed 
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to describe Xf to Decoder 1. Decoder 2 knows Xf via the crib, it then needs to infer the unique bin 
index which was sent, as then it would infer X^. The only issue to verify is that the Xf codeword known 
via cribbing should not lie in two bins. We upper bound the probability of occurrence of such an event by 
2n{i{X;XuX2)-H(Xi))^ ^^ jj^gj.^ ^g overall 2"^('^'^i'^2) Xf codewords, and the probability that a particular 
X" lies in two bins is 2^"^(^i). This event has a vanishing probability so long as I{X; Xi, X2) < H{Xi). 
Thus the achieved rate region is Ri > I{X;Xi,X2) such that the constraint I{X; Xi,X2) < H{Xi) and 
distortion constraints are satisfied. 

The general coding scheme when i?o > is depicted in Fig. 7 and has a "doubly-binned" structure. 
Non-zero Rq helps reduce Ri by providing an extra dimension of binning. We first generate 2"'^^'^'^'^' 
X2 codewords, the indexes of which are the rows (or horizontal bins), and then in each row, we generate 
2ni{X;Xi\X2) j^n codewords. For each row, these X^ codewords are then binned uniformly into 2"^" 
vertical bins, which are the columns of our "doubly-binned" structure. Thus each bin is "doubly-indexed" 
(row and column index) and has a uniform number of 2"'^^^^'^^\^^^~-^°'> Xf codewords (as in Fig. 7). 
Note that this extra or independent dimension of vertical binning was not there when Rq = 0. Intuition is 
that column indexing with common rate Rq is independent or orthogonal to the row indexing, and hence 
it helps to reduce the private rate Ri. The column or vertical bin index is described to both the decoders 
via common rate Rq and thus Ri reduces to I{X; Xi, X2) — Ro to describe X" to Decoder 1. Here again, 
from knowledge of the crib, Xf and the column index. Decoder 2, infers the unique row index, which 
now will require I{X; Xi,X2) - Ro < H{Xi). 

We now describe the achievability in full detail. 

• Codebook Generation : Fix the distribution P^ x x ' ^ -^ ^ ^^'^'^ ^^^^ E[di{X,Xi)] < j^ and 
E[d2{X,X2)] < j^- Generate codebook Cj^ consisting of 2"^("^'^2) x^^nih) codewords generated 
i.i.d r^ Pj^ , rrih G [1 : 2'^^^^''-^'^']. For each ruh, first generate a codebook C^ {mh) consisting of 
2ni{X;Xi\X2) j^n codewords generated i.i.d. ~ Pj^ ,j^ , then bin them all uniformly in 2"^° vertical 
bins B{my), niv e [I : 2"^«] and in each bin index them accordingly with / G [1 : 2"(-f(^;^il^2)-^«)]. 
As outlined earlier, rrih corresponds to the row or horizontal index and niv corresponds to the column 
or vertical index in our "doubly-binned" structure, while I indexes Xf codewords within a "doubly- 
indexed" bin. Thus for each row and column index pair, (171^,171^), there are 2"(^("'^'^il^2)--Ro) xn 
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2n{i{X;Xi\X2)-Ro) ^jj codcwords in each "doubly-indexed " bin. 
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Fig. 7. "Double Binning" - achievability scheme for the non-causal perfect cribbing. 
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codewords. X^ can therefore be indexed by the triple {mh,my, I). The codebooks are revealed to the 
encoder and both the decoders. 

Encoding : Given source sequence X", first the encoder finds m^ from C^ such that (X", ^^'(ra/j)) G 
7^". Then the encoder finds pair (m„,/) such that (X",X"(?n,/i,mt,, /),X2'(m/i)) G 7^". Thus 
Xi{mh,my,l) G B{mv). Encoder describes column or vertical bin index m^ as Rq to both the 
decoders, and the tuple {mh,l) to the Decoder 1 as rate Ri. Thus 



Ri > I{X- X2) + I{X- X1IX2) - i?o = I{X- Xi, X2) - i?o. 



(11) 



Decoding : Decoder 1 knows all the indices {mh,mv,l), and it constructs X" = X^{mh,my,l)- 
Decoder 2 receives Xf from the non-causal cribbing and it also knows the column index m^ through 
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rate Rq. It then checks inside the column or vertical bin of index m,^, to find the unique row or 
horizontal bin index nih such that Xf = X]'{mh,'m^,l) for some [ G [1 : 2"('^(^;^il^2)-i?o)] jhe 
reconstruction of the Decoder 2 is then X2 = -^2^("ife)- 
Distortion Analysis : Consider the following events : 

1) 

£1 = No X2 is jointly typical to a given X"- (12) 

= |(X-,X2"(m;,))^r,",Vm;,G[l:2"^(^;^^)]|. (13) 

The probability of this event vanishes as there are 2'^^^-^''-^^' X2 codewords, (cf. Covering 
Lemma, Lemma 3). 
2) 

£2 = No Xf is jointly typical to a typical pair {X^,X^) (14) 

{X^,XS{mh))eTr 
nUx^,Xnmh,m„l),X^imh)) ^ 77,V m, G [1 : 2"^"], 

V / G [1 :2"W^'-^il-^^)--f^'')]l 

(15) 

The probability of this event vanishes as corresponding to each m/j there are 2"^(^'^il"'^2) xn 
codewords, (cf. Covering Lemma, Lemma 3). Without loss of generality, now suppose that 
encoder does the encoding, {mh,my,l) = (1,1,1). Decoder 2 receives Xf via non-causal 
cribbing. The next two events are with respect to Decoder 2. 
3) 

£■3 = Xf does not lie in bin indexed by rrih = 1 and rriy = 1 (16) 

X{' / lf(l,l,/"),V le [1 : 2"W^;^il^^)-«")]\. (17) 



But the probability of this event goes to zero, because due to our encoding procedure, X^ 
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^l"(l,l,l) 



4) 



£4 = Xi lies in a bin with row index, rhh / 1 and column index my = 1. (18) 

X]^ = X^{mh, 1,1), Thh / 1 for some I £ [I : 2"(-f(^;^il^2)-«o)] j. 

(19) 

Since X^ = Xf{l, 1,1), this event is equivalent to finding Xf lying in two different rows or 
horizontal bins, but with the same column or vertical bin index (m^ = 1). The probability of 
a single Xf codeword occurring repeatedly in two horizontal bins indexed with different row 
index is 2~"-^^'^^\ while knowing the column index, rriy, total number of Xf codewords with a 
particular column index are, 2"(^("'^'"^i'^2)-fio)^ ^q t^g probability of event £4 vanishes so long 
as. 



I{Xi;Xi,X2)-Ro<HiXi 



(20) 



Thus consider the event, <S = <Si U <S2 U <S2 ^£4, using the rate constraints from Eq. (11) and Eq. (30), 
the probability of the event vanishes if, 



Ro + Ri > I{X;Xi,X2) 

Ro > {I{X;Xi,X2)-H{Xi)}+. 



(21) 
(22) 



We will now bound the distortion. Assume without loss of generality that, dj(-, •) < D^ax, for i = 1, 2. 
For both the decoders, {i = 1,2), 



E 



d{X\Xf 



= P{£)E d{X'\X'^)\£ +P{£^)E d{X'\X'^)\£ 

< P{£)D^ax + {l + e)E[d{X,Xi)] 

< P{£)D^ax+Di, 



(23) 
(24) 
(25) 



where (a) is via typical average lemma (cf. Typical Average Lemma 2). Proof is completed by letting 
n — > 00 when P{£) — ^ 0. 
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Converse : Converse for this setting follows by substituting Zi = Xi in the converse for the deterministic 
function cribbing in the next subsection. 

Note 1 (Joint Typicality Decoding). Note that here our decoding for Decoder 2 relies on finding a unique 
bin index in which X^ (obtained via cribbing) lies, and there is an error if two different bins have the same 
X^. An alternative based on joint typicality decoding can also be used to achieve the same region as follows 
: Decoder 2 receives X^ via non-causal cribbing and it also knows the column index rriy through rate Rq. 
It then finds the unique row or horizontal bin index m/j such that (X", X"(?tt-/j, ttt,^, /),X^(?Ti/j)) G 7^" 
for some [ G [1 : 2''''-'^^^^'^^^^^^~^'>^]. The reconstruction of the Decoder 2 is then XJ^ = X^{mh). We 
analyze the following two events, assuming without loss of generality that encoder does the encoding 
{mh,m^,l) = (1,1,1). 



£d,i = Decoder 2 finds no jointly typical X^ indexed by nih = 1 and ra^ = 1 (26) 

= |(lf,li"(l,l,/"),X^Xl)) i V'y ^"e [1 : 2"(^(^;^il^^)-^'')] j. (27) 

But the probability of this event goes to zero, because due to our encoding procedure, with high proba- 
bility, (X", Xf (1, 1, l),X^{l)) G r/'. As Xi" = X{'(1, 1, 1) this implies, (X^", If (1, 1, 1), 1^(1)) G 

'-j—n 
' e • 

Sd,2 = Decoder 2 finds a jointly typical X^ codeword in row with index, rhh i^ 1. (28) 

= Ux^,X1[mh,\;i),Xl{mh)) G 7^", ih^ + \ for some [ G [1 : 2"W^-^^I^^)-«»)]|. 

(29) 

By Lemma 4 (Packing Lemma , substitute, \A\ = 2"(^('^;^i'^2)-Ro)^ U = (l),X = {X2, Xi),Y = Xi), 
probability of this event goes to zero with large n, if 

IiXi;Xi,X2)-Ro < I{Xi;Xi,X2) = H{Xi). (30) 

Thus we obtain the same constraint with the joint typicality decoding for Decoder 2. In all the subsections 
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to follow, for Decoder 2, joint typicality decoding can also be used as an alternative to the decoding that 
will be described. 

■ 
2) Deterministic Function Cribbing: 

Theorem 2. The rate region TZ{Di,D2) for the setting in Fig. 8 with deterministic function (non-causal) 
cribbing is given as the closure of the set of all the rate tuples {Ro,Ri) such that, 



Ro + Ri > I{X;Xi,X2) 

Ro > {I{X;Zi,X2)-H{Zi)}+, 



(31) 
(32) 



for some joint probability distribution PxPz x \x-^x \z x x ^^^^ ^^'^^ E.[di{X , Xi)] < Di, for i = 1,2. 



Xn 



ENCODER 



Ti{X") e{l: 2"'^'} 



DECODER 1 



Xf(ro,ri),A 



To(X") € {1 : 2"'^"} 



Zi.i = .9(^1,0 



J ^ X2,{To,Z^),D2 

DECODER 2 *- 



Fig. 8. Successive refinement, with decoders cooperating via (deterministic function) non-causal cribbing. 



Proof: 
Achievability : The scheme is similar to the achievability in the previous section, where cribbing was 
perfect, with some minor differences. We give an outline here and highlight the differences, deferring the 
complete proof to Appendix C. The codebook here also has a "doubly-binned" structure as in Fig. 7, 
the difference being that each "doubly-indexed" bin has a uniform number of Z" codewords instead of 
X^. So first 2"'^('^'^2) X^ codewords are generated, for each of them, 2"^(^'^il'^2) Zf codewords are 
generated, which are then vertically binned uniformly into 2"^" vertical bins (columns). Then for each 
^n^ 2ni{X;Xi\Zi,X2) j^n codewords are generated. Here also, the column index is described as Rq and the 
remaining indices are described as Ri, which hence is equal to I{X; Xi, Zi, X2) — Rq = I{X; Xi, X2) — 



17 

Rq. Decoder 1 can, as usual, construct its estimate since it knows all the indices. Decoder 2, infers the 
row index from the deterministic function crib, Zf and knowledge of the column index. The decodability 
of a unique row index depends on the fact that there should not be the same Zf codeword in two rows. 
This requires (as we saw in the previous section), I{X;Zi,X2) — -Ro < H{Zi). 

Converse : Assume we have a (2"^", 2"^^ , n) code (as per Definition 4) achieving respective distortions 
Di and L>2. Denote Ti = /i,n(^") and To = /2,n(^"). Consider, 

H{Z^,To) > IiX^;Z^,To) (33) 

^=^ I{X^;Z^,X^,To) (34) 

> I{X^;Z^,X^) (35) 

n 

= 5]/(X,;Zr,X2"|X^-i) (36) 

i=l 
n 



^I{X,;Zl\X^,X'-') (37) 

i=l 
n 

> Y,I{X,■,Zl,i,X2,^) (38) 



j=l 



= nV-/(X,;Zi,„l2,^) (39) 

i=l 

^^ nI{XQ;Z,,Q,X2,Q\Q) (40) 

^^ nI{XQ;Z,^Q,X2,Q,Q) (41) 

> nI{XQ;Z,,Q,X2,Q) (42) 

H{Z{\To) < H{Z{')+H{To) (43) 

n 

< ^H{Zi,i) + nRo (44) 
1=1 

= nH{Zi^Q\Q) + nRo (45) 

< nH{Zi^Q) + nRo (46) 

n{Ro + Ri) = H{To,Ti) (47) 

= H{To,T,)-H{To,Ti\X^) (48) 



= /(X";ro,Ti) 

^^ I{X^;n,Ti,X^,X^) 

n 

= 5]/(X,;To,ri,li",l2"|X*-i) 

i=l 

n 

i=l 

n 

> / ,I{Xi]Xi^i^X2,i) 

i=l 
n 

= ny^-I{Xi■,X^,^,X2,i) 

i=l 

= nI{XQ;Xi,Q,X2,Q\Q) 
= nI{XQ;Xi,Q,X2,Q,Q) 

> nIiXQ;X,^Q,X2,Q), 
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(49) 
(50) 
(51) 

(52) 

(53) 

(54) 

(55) 
(56) 
(57) 



where (a) follows from the fact that X2 is a function of (To,Z"), (b) follows from the independence of 
Xi and X^~^, and (c) follows by defining Q G [1 : n] as a uniformly distributed time sharing random 
variable independent of the source, (d) follows from the independence of Q with the source process, (e) 
follows as (X",X^) is a function of (To,Ti) and finally (f) follows similarly from the independence of 
Xi and X'-^^. Finally, we bound the distortion as. 



Di > ^ 

= E 



[d{X\X2) 
\J2d{X,,X, 



i=l 



E[d(XQ,l,,Q)]. 



(58) 
(59) 
(60) 



The proof is completed by noting that the joint distribution of {Xq,Xi^q,X2^q) is the same as that of 

{X,Xi,X2). ■ 

Note 2. Due to the structure of our problem, i.e., Z\ = g{Xi), it is easy to prove the Markov relation, 
(X, X2) — Xi — Z\, hence the distribution mentioned in the statement of the theorem, can equivalently 



be factorized as, PxP 



Xi,X2|X-^{Zi=s{Xi)} 
theorems to follow, and we omit this explanation henceforth. 



, (which is the form stated in Table I). This applies similarly for 
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B. Strictly-Causal Cribbing 
1) Perfect Cribbing: 

Theorem 3. The rate region TZ{Di, D2) for the setting in Fig. 9 with perfect cribbing (strictly causal) is 
given by the closure of the set of all the rate tuples (Rq , Ri ) such that, 



Ro + Ri > IiX;Xi,X2) 

Ro > {I{X;Xi,X2)-H{Xi\X2)y 



for some joint probability distribution P 



X,Xi,X, 



such that E[di{X, Xj)] < Di, for i = l,2. 



(61) 
(62) 



X" 



ENCODER 



Ti{X")e{l: 2"-^'} 



DECODER 1 



X^{To,Ti),Di 



ToiX") e {I: 2"-^'>} 



DECODER 2 



X2,i{To,Xl ),D2 



Fig. 9. Successive refinement, with decoders cooperating via (perfect) strictly-causal cribbing. 



Proof: 
Achievability : 
We will show the achievability of the following region instead, 

R^ + Ri > I{X-Xi,U) 

Ro > {I{X;Xi,U)-H{Xi\U)}+, 



(63) 
(64) 



for some joint probability distribution P^ x u'^\x =f(u)\- ^^^^ ^^^^ ^^^ ^^^^ region in the theorem will 
then be obtained by simply taking U = X2- Here we deliberately present our encoding scheme with an 
auxiliary random variable as this will be used (with minor changes) to derive the achievable region for the 
case of causal cribbing discussed in the next subsection. 
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"Forward Encoding" and "Block Markov Decoding" scheme : 

We use a new scheme that we refer to as "Forward Encoding" and "Block Markov Decoding". We first 
briefly give an overview of the coding scheme and for simpUcity consider the case when common rate 
i?o = 0. Thus the source description is available only to Decoder 1, while Decoder 2 has access to the 
reconstruction symbols of Decoder 1, but only strictly-causally. Hence in principle we cannot deploy a 
scheme to operate in one block as was done for non-causal cribbing. We need to use a scheme to operate in 
multiple (large number) of blocks, and use an encoding procedure where Xf of the previous block carries 
information about the source sequence of the current block. In this way due to strictly causal cribbing, 
in the current block. Decoder 2 will know all the reconstruction symbols of Decoder 1 from the previous 
block, which will contain information about the source for the current block. This is the main idea and is 
operated as follows : in each block, first we generate X'-^i^''^) (J^ codewords, and for each [/" codeword, 
we generate 2"^^"'^'^) bins and in each bin 2"^("'^'^il^) X^ codewords are generated. In each block, f7" is 
jointly typical with the source sequence in the current block and the bin index describes the C/" sequence 
jointly typical with the source sequence of the future block. This bin index carries information about 
the source in the future block. Hence, we address encoding as "Forward Encoding". Decoding is "Block 
Markov Decoding", as it assumes both decoders have currently decoded the [/" sequence of the previous 
block. The bin index and index of the X^ codewords is described as Ri which hence is taken to be 
I{X; U) + /(X; Xi\U) = I{X; Xi,U). Due to cribbing. Decoder 2 knows the Xf of the previous block 
and aims to find the bin index in which it lies. And as we argued in previous sections, this is possible if 
I{X;Xi,U)<H{Xi\U). 

The general scheme when i?o > is depicted in Fig. 10. The additional step which we add to the 
description above (for Rq = 0) is to bin in an extra dimension, i.e., with respect to each f^" sequence we 
generate a "doubly-binned" codebook (as in the achievability of non-causal cribbing, cf. Fig. 7). The row 
index encodes f7" sequences of the future block and Xf codewords for each row are uniformly binned 
into 2"^" columns. The column index is the common description to both decoders, so Ri reduces to 
I{X; Xi, U) — Ro, and the decodability of Decoder 2 requires the condition liX; Xi, U) — Rq < H{Xi\U). 

We now explain this coding scheme in detail and how it helps establish the achievable region when the 
cooperation between the decoders is via strictly causal cribbing. 
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1) Codebook Generation : The scheme does compression in blocks. Fix the number of blocks to be B. 

In each block, n source symbols are compressed. Fix a joint probability distribution, P^^^ ^ = 

^u X X -'-jx =f(U)\ ^^^ some function / and e > such that ^[di{X, Xi)] < ^ and E[d2{X, X2)] < 
D2 

Now in each block we generate codebook as follows. First we generate a codebook Cu{b) = 
{u"(6,m) ~ nr=i Pu{ui{b,m)),m= [1 : 2"^(^;'^)]} for each block 6 G [1 : B]. For each u"(6,m), 
we create 2"-^^^'^^ horizontal bins or rows B{mh) which are indexed as m/j G [1 : 2^^^^'^^]. In 
each bin we generate a codebook 2"^(^'"'^il'^) X^ codewords which are then binned again into 
2"-Ro vertical bins or columns, B{my) uniformly, m^ G [1 : 2"^"] and index them accordingly by 
/ G [1 : 2"-^^^^'-^^'^>~^''>]. Thus X" can be equivalently indexed as the tuple (6, m, rrih, m^, I). Hence 
for each u" as explained earlier we have a "doubly-binned" structure, nih denotes the row index and 
rriv denotes the column index. The codebooks are then revealed to both the encoder and decoders. 

2) Encoding : X"^ is known to the encoder. From now on additional subscripts will stand for block 
index, eg. m^ 2 means the row index in block 2, or mv,2 means the column index in block 2. Also 
additional scripts in parenthesis would denote the sequence in a block, eg. X"'{b) will stand for the 
source sequence in block b, Xf{b) stands for reconstruction of Decoder 1 in block b. Encoding is 
as follows : 

a) For the first block, 6 = 1, assume mi = 1. Encoder then finds index 7712, such 
that (X"(2),t/"(2,m2)) G 7^". The encoder then looks in the codebook Cu{l) to 
find [/"(!, mi). Then it looks in the row or horizontal bin indexed by ruh^i = m2 
corresponding to the found [/"(I, mi), and finds the index tuple (mt,i,/i) such that 
(Xi"(l,mi,m/,,i,m„,i,Zi),X"(l), t/"(l,mi)) G 7^". As found X'-^ G S(m^,i), the index tuple 
(r7T,/ji,/i) is described as Ri and rriv^i is described as Rq. 

b) In the block 6 (G [2 : B — 1]) encoder knows nih from encoding procedure in previous block 
such that (X"(6),[/"(6,m6)) G 7^". It then finds index mt+i such that (X"(6+ 1),?7"(6 + 
l,mfe+i)) G 7^". Now the encoder identifies the codeword, U'^{b,mh), from the codebook 
Cjj{b), looks in the corresponding row or horizontal bin indexed as rrihi, = m-t+i and finds the 
index tuple {m^fijb) such that {Xf{b,mb,'mh^i„my^hJb),X"-{b),U^{b,mi))) G 7^". As found 
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Fig. 10. "Forward Encoding" and "Block Markov Decoding" - achievability scheme for the strictly-causal perfect cribbing. 



Xf e B{my^h), the index tuple {rrihfijb) is described as Ri and m^^b is described as Rq. 
c) In the last block b = B, the encoder knows ttt,^ from encoding procedure in the previous 
block. Fix niB+i = 1- Encoder identifies U^{B,7nB) from the codebook Cu{B), looks in the 
corresponding row or horizontal bin mh^B = "tub+i and finds the index tuple {rriy^BjB) such 
that {Xf{B,mB,mh,B,my^B,lB),X''{B),U'''{B,mB)) G TJ". As found Xf e B{m^^B), the 
index tuple {mh,B,lB) is described as Ri and ttt-^^s is described as Rq. 



Hence the encoding has a "Forward Encoding" interpretation, as we encoded the source sequence of 
the future block as the row or horizontal bin index of the "doubly-binned" codebook in the present 
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block. As at each block b, Ri encodes for {nih^bJb), thus 

I{X;U) + IiX;Xi\U)-Ro<Ri. (65) 

3) Decoding : Decoding for both the decoders is as follows : 
Decoder 1 

a) For the first block h = \, Decoder 1 knows mi = 1, and since it knows the index (m/^i, 771^,^1, h) 
it identifies ^"(1) = X^{1, mi, 111^^1,111^^1,11) as its source estimate for the first block. 

b) For the block 6 (G [2 : B]), Decoder 1 knows m^ from the index sent by the encoder in the 
(6 — 1) block (as m^^^^^i = mi,) and since it knows the index {mh^b:^^v,b,h) for the current 
block, it identifies Xf(6) = X]^{b,mb,mh,b7'>T^v,b,lb)y as its source estimate. 

Decoder 2 

a) For the first block 6=1, Decoder 2 assumes rhi = 1 and generates its estimate ^^(1) = 
/(t/"(l>i)). 

b) For the block 6 (G [2 : B]), Decoder 2 has already estimated rhb-i in 6 — 1 block. It 
also knows X^{b — l)(because of strictly causal cribbing) and m^fi-^i through Rq. It then 
looks into the vertical bin with index m^^b-i in the codebook corresponding to the codeword 
C/"(6 — 1, rhb-i), and finds a unique row or horizontal bin index rhh^b-i such that Xf{b— 1) = 
Xi{b- l,mb-i,mh,b-i,m^,b-ijb~i) for some h-i G [1 : 2"('^(^;^il'^)"^«)]. But note that 
estimating rhh^b-i is equivalent to estimating ifib, because of our forward encoding procedure, 
thus Decoder 2 constructs its source estimate for the block b as ^^(6) = f{U''^''{b,mb)). 

Decoding has a "Block Markov Decoding" interpretation as we see that the decoding for both 
decoders relies on what was successfully decoded in the previous block. 

4) Rate Region and Bounding Distortion : We assume without loss of generality, di{-, •) < Dmax < oo> 
i = 1, 2. In the encoding and decoding scheme, mi was chosen to be a fixed value, deterministically 
chosen prior to the compression, agreed upon by both encoders and decoders. Hence, for both the 
decoders distortion in general will not be met for the first block, however we are generous enough 
to allow for maximum distortion for the first block, which will eventually have insignificant impact 
on total distortion as the number of blocks becomes large. Consider the following encoding and 
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decoding events which will help to bound the distortion at Decoder 1 and Decoder 2. Suppose in 
block b — 1 and b, index tuples {mh^b-ijh-i) and {mhfi,lb) are described by the encoder to the 
Decoder 1, and that mh^b-i = f^b and ruh^b = "mb+i, V 6 = [2 : B]. 
a) Encoding Events : 



£e,i{b) = No f7" sequence is jointly typical with source in block b (66) 

(X"(6), U'\b,mb)) i r," V m, G [1 : 2"^(^'^)]1, (67) 



for 6 = [2 : S]. By Covering Lemma 3, the probability of this event goes to zero as there are 

^ai(x-fj) jjn codewords Similarly, P{£e,i{b+l)) -^ 0. Suppose, (X"(6), t/"(6, ^5)) G TJ" 
and (X"(6 + 1), U^{b + 1, nib+i)) G 7^", thus row index in block b is mh^b = ^b+i- 

£e,2{b) = No X" sequence is jointly typical with the typical pair {X, U) in block b 

= Sl,{b)r^si,{b + i) 

r\\ {X^{b,mb,mh,b,rh^,bJb),X''{b),U''{b,mb)) iT^y tuples {m^^b,ib) [, 

(68) 

for 6 = [1 : i?], where, 

£l^{b) = |(X"(6),C/"(6,m,))G7;"| (69) 

£li{b+l) = |(X"(6 + l),C/"(6+l,m5+i))G7r|. (70) 



By Covering Lemma 3, this event has vanishing probability as for every row index there are, 

2ni(X;Xi\u) xn codewords. 

b) Decoding Events : Decoder 1 can perfectly construct the Xf{b) sequences in the block b. 
Decoder 2 in block b knows X"(& — 1). For the Decoder 2, for 6 G [2 : B], assume it has 
decoded correctly the message, 7fib-i = nib-i in the 6—1 block and the encoder sends the 
row index m^b-i = 'f^b in the block 6 — 1 to Decoder 1. Also Decoder 2 knows m^b-i 
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through Rq. Decoder 2 needs to find an estimate mh,b-i, or equivalently an estimate of m^ (as 
fnh,b-i = rnb)- Consider the following events : 



Sd,i = Xi{b — 1) does not lie in row with index, rrih^-i = rrib and column index my^-i 
Xi"(6-l) = lr(6-l,mfe_i,m;,,b„i,m,,6_i,4_i)|, (71) 



V / G [1 : 2"''^("'^'^il^2) ^"']. But the probability of this event goes to zero, because due to 
our encoding procedure, X^{b— 1) = X"(6— l,mb-i,mhfi-i,my^b-i,lb-i)- 

£d,2 = Xf{b — 1) lies in a row with index, m/^ f,_i / nih and column index niyh^i 

Xi{b-1) = Xi{b-l,mb-i,'ihh,b-i,'my^b~iJb~i), rhh^b-i^mb), (72) 



for some I e [I : 2"(^(^''^il'^2)~^°)]. This event is equivalent to finding Xf (6 - 1) 
corresponding to U"'{b — l,r7T,5_i) lying in two different rows or horizontal bins, but with 
the same column or vertical bin index (m^, f,_i). The probability of a single X^ codeword 
(corresponding to a [/" codeword) occurring repeatedly in two horizontal bins indexed with 
different row index is 2^'^^^-^^^^', while knowing the column index, total number of Xf 
codewords with a particular column index are, 2"(^("''-'"^i'"^2)-%)^ ^q t^e probability of event 
£d,2 vanishes so long as, 

I{Xi-Xi,X2)-R^<H{Xi\U). (73) 

Thus consider the event £{b) = £e,i{b) U £e,2{b) U £d,i{b) U £d,2{b)- We have, 

P{£{b)) < P{£e,i{b)) + P{£e,2{b)) + Pi£d,im + P{£d,2m, (74) 

which vanishes to zero with large n, for each block 6 = [2 : i?], if [from Eq. (65), Eq. (73)], if, 

Ro + Ri > I{X;Xi,U) (75) 

Ro > {I{X;Xi,U)-H{Xi\U)}+. (76) 
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We will now bound the distortion. The distortion for both the decoders in the first block is bounded 
above by Dmax- Consider the block b = [2 : B] for Decoder 1, 



di(X"(6),Xi"(fe)) 



(a) 



P{S{h))E[di{X^{b),X^{h))\8{b) 

+ P{£'{h))^ [di{X'\h),X^{h))\£^{h)\ {11) 

< P{£{h))D^ax + P{£\h)){l + e)E[di{X,Xi)] (78) 

< P{£{b))Drnax + P{£%b))Di, (79) 

where (a) follows from Typical Average Lemma 2, as given £'^{b), (X"(5),X"(6)) G 7^". Thus as 
n — )■ oo, P{£{b)) — )■ 0, hence the distortion is bounded by Di in block b. Similarly for Decoder 2, 

'd2{X^{b),Xl^{b))\ = P{£{b))E[d2{X^{b),Xl^{b))\£{b) 

+P{£\b))E [d2{X^\b),Xl^{b))\£\b)\ (80) 

(6) 

< P{£{b))Draax + P{£%h)){l+e)£[d2{X,X2)] (81) 

< P{£{b))Draax + P{£\h))D2, (82) 

where (b) follows from Typical Average Lemma 2, as given £^{b), (X"(6), t/^^'(6, mf,)) G 7^", and 
since XJ^{b) = f{W{b,mi,)), (X"(6),X^(6)) G TJ". Thus the distortion is bounded by D2 in block 
b. The total normalized distortion in B blocks for Decoder 1 and Decoder 2 is bounded above by 



hDmax + ^^TT^i and j^Djnax + ^^T^D2 respectively. Proof is completed by letting, B 



B' 



CX). 



Converse : Converse in this subsection is skipped and follows from the converse of deterministic function 
cribbing of the next subsection, by the substitution Z\= X\. ■ 

2) Deterministic Function Cribbing: 

Theorem 4. The rate region TZ{Di,D2) for the setting in Fig. 11 with deterministic function cribbing 
(strictly causal) is given as the closure of the set of all the rate tuples {Rq,Ri) such that, 



Ro + Ri > I{X-Xi,X2) 

Ro > {I{X;Zi,X2)-H{Zi\X2)V, 



(83) 
(84) 
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for some joint probability distribution PxPy x \x^' 



Ai,X2\y^ Xi\^i,yC2,X 



such that E[di{X, Xi)] < Di, for i = 1,2. 



Xn 



ENCODER 



Ti{X") e{l: 2"'^'} 



DECODER 1 



Xf(ro,ri),A 



To{X") e{l: 2"-^"} 



DECODER 2 



Zi,i = g{Xi^t) 



X2,(To,iri),D2 



Fig. 11. Successive refinement, with decoders cooperating via (deterministic function) strictly-causal cribbing. 



Proof: 

Achievability : 
The extension to deterministic function cribbing from perfect cribbing follows similarly to the case of 
noncausal cribbing in Section III-A2. We omit the details of achievability and describe the key idea. 
Here also, achievability is first proved with auxiliary random variable U and the following region will be 
achieved. 



Ro + Ri > IiX;Xi,U) 

Ro > {I{X;Zi,U)-H{Z,\U)y 



(85) 
(86) 



for some joint probability distribution PxPz u\x^x \z ux'^\x =f(U)\ ^^^^ ^^^^ ^[di{X,Xi)\ < Di, for 
i = 1,2. The codebook structure remains almost the same, just that instead of (uniformly) binning Xf 
into vertical 2"^" bins, as done in the setting of the previous subsection with perfect cribbing, we bin Zf 
codewords and X" codewords are then generated on the top of each Zf codewords. Encoding changes 
accordingly and Decoder 2 tries to infer the row index from the deterministic crib which it obtains from 
Decoder 1. 

Converse : Assume we have a (2"^« , 2"^^ , n) distortion code (as per Definition 4) such that 
{Ro,Ri,Di,D2) tuple is feasible (as per Definition 2). Denote Ti = /i,n(^") and Tq = /2,n(^")- 
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Identify the auxiliary random variable Ui = {Tq^Z\ 



i-l\ 



H{Z^,To) > I{X^;Zl\To) (87) 

n 

= Y.I{Xi;Z^,To\X'-^) (88) 

n 

^=^ Y,HXf,Z^,n,X'-') (89) 



i=l 
n 



> J]/(Xi;Zi,To) (90) 

n 

= J]/(Xi;4i,t/.) (91) 

1=1 

> nI{XQ;Zi,Q,UQ) (92) 

n 

H{Z^,To) = J]ii-(4i|ro,Zi-i) + i7(To) (93) 

i=l 
n 

< J]i?(4i|f/,) + ni?o (94) 

< nH{Z,,Q\UQ)+nRo, (95) 

where (a) follows from the independence of Xi with X"^^^ and Q £ [1 : n] is similarly defined 
an independent (of source) uniformly distributed time sharing random variable. As argued in previous 
subsection of perfect cribbing, we lower bound n(i?o + ^i) with nI{X; Xi^q,Uq). Note that as 
^2,Q = /{Uq), for some function /. Lastly we bound the distortion for both decoders as we did in 
previous section and note that the joint distribution of {Xq,Xi^q,X2,q) is the same as {X,Xi,X2) to 
derive the rate region with auxiliary random variable. It is easy to see that in inequalities (91) and (94), 
we can replace Ui with X2,i and this helps to provide converse for the region without auxiliary random 
variable provided in the theorem. ■ 



C. Causal Cribbing 

1) Perfect Cribbing: 
Theorem 5. The rate region TZ{Di, D2) for the setting in Fig. 9 with perfect causal cribbing that is X2^i 
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is a function of {Tq,XI), is given as the closure of the set of all the rate tuples {Rq,Ri) such that, 

Ro + Ri > I{X;Xi,U) (96) 

i?o > {I{X-Xi,U)-H{Xi\U)}+, (97) 

for some joint probability distribution P^ x u^ix =f(ux )\ ^'^^^ ^^^ ^[di{X,Xi)] < Di, for i = 1,2 

and \U\ < \X\ \Xi\ +4. 

Proof: The achievability remains the same as in strictly causal cribbing, in terms of encoding and 
decoding operations at Decoder 1. For Decoder 2, the only change is in constructing ^^(6) for each block, 
which in this case is constructed as, X2^{h) = f{Ui{b,mi,),Xi^i). The steps in the converse are exactly 
the same as in the strictly causal cribbing case, except that this time we identify X2,Q = f{UQ,Xi^Q). 
The cardinality bounds on U follow standard arguments as in [21] : W should have \X\ \Xi\ — 1 elements 
to preserve the joint probability distribution P^ ^ , one element to preserve the markov chain, {X, Xi ) — 
U — X2, two elements to preserve the mutual information quantities, I{X]Xi,U) and {I{X;Xi,U) — 
H{Xi\U)}^ and finally two more elements to preserve the distortion constraints. ■ 

2) Deterministic Function Cribbing: 

Theorem 6. The rate region TZ{Di, D2) for the setting in Fig. 11 with deterministic function cribbing but 
with causal cribbing, that is, X2^i is a function of {Tq,XI), is given as the closure of the set of all the 
rate tuples {Rq , Ri ) such that, 

Ro + Ri > I{X-Xi,U) (98) 

i?o > {I{X-Zi,U)-H{Zi\U)]+, (99) 

for some joint probability distribution PxPz u\x^x \z ux'^\x =f(uz )\ ^^^^ ^^'^^ E[(ij(X, Xj)] < Di, 
for i = 1,2 and \U\ < \X\ \Xx\ +4 

Proof: The achievability remains the same as in strictly causal deterministic function cribbing, in 
terms of encoding operation and decoding operation at Decoder 1. For the Decoder 2, only change is in 
constructing Xl^ili) for each block, it is constructed as, X^iifi) = f{Ui{b, mi,), Zi^i). The steps in converse 
are exactly the same as in strictly causal cribbing case except that we identify, ^2,Q = /{Uq, ^i,q)- ■ 
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IV. Special Cases 
In this section, we study some special cases of our setting and also compute certain numerical examples. 

A. The Case Rq = 



X" ^ T{X") e {I : 2"^^} , . Xi"(r),Di 

ENCODER ^^ nOECODER 1^ n 



Z\.i = g{Xi^,) 



-^ ^ X2AZf),D, 

DECODER i *- 



Fig. 12. Special case of successive refinement with cribbing decoders, when the common rate is zero. Here again d — n, d — i — 1 
and d — i respectively stand for non-causal, strictly-causal and causal cribbing. 



One special yet important case of the setting studied in previous sections, is that when i?o = as shown 
in Fig. 12. Here the encoder describes the source to only Decoder 1, while Decoder 2 attempts to find the 
reconstruction of the source within some distortion via cribbing reconstruction symbols of the Decoder 1 , 
non-causally, causally or strictly causally. Table II provides the minimum achievable rate (R = R{Di, D2)) 
for various cases, derived when Rq = 0, using Theorem 1 through 6. Distortion constraints are omitted 
for brevity. 

B. Null g function 

Our expressions reduce to the successive refinement rate region (cf. Equitz and Cover [18]), when g is 
a trivial function. To see this consider rate region for non-causal cribbing with deterministic cribbing (cf. 
Theorem 2), as given below, 

Rg + Ri > I{X-Xi,X2) (100) 

i?o > {I{X-Zi,X2)-H{Zi)}+, (101) 
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R{Di,D2) 



Non-Causal (d = n) 



Strictly-Causal {d = i-l) 



Causal {d = i — 1) 



Deterministic min I[X ; Xi , X2 ) 

Function s.t. I{X; Zi, X2) < H{Zi) 

Cribbing 

(p.m.f.):P{X,Xi,X2)x 

^{Z^=f{X^)} 



mmI{X-Xi,X2) 

s.t.I{X;Zi,X2)<H{Zi\X2) 

(p.m.f.):P{X,Xi,X2) 

^{Z^=f{X^)} 



mmI{X;Xi,U) 
s.t. I{X;Zi,U) <H{Zi\U) 

(p.m.f.):P{X,Xi,U)x 

l{Zi=/(Xi),X2=/(Zi,t/)} 



TABLE II 

Results for the Successive Refinement with Cribbing Decoders, when common rate, Rq = 0. 



If g is null, Zi is constant and hence the region reduces to, 

Ro + Ri > I{X-Xi,X2) 
Ro > I{X;X2), 



(102) 
(103) 



for some joint probability distribution Px x x ^^^^ that E[di{X,Xi)] < Di, for z = 1,2, which is also 
derived in Equitz and Cover [18]. 



C. Numerical Examples 

We provide an example illustrating the rate regions of non-causal and strictly causal cribbing. Along 
with them, the region without cribbing is also compared. The rate regions for these three cases from the 
theorems in the paper are shown in the Table III. Distortion constraints are omitted for brevity. 

We plot for a specific example (cf. setting in Fig. 4 with perfect cribbing) with a bernouUi source 
X r^ Bern(0.5), binary reconstruction alphabets and hamming distortion. We consider a particular distortion 
tuple {Di,D2). Due to symmetry of the source, for the optimal distribution, it is easy to argue that, 
Px X |x(^i'^2|2:) = P^ -^ ,x{xi,X2\x), where x stands for complement of x. Thus all the expressions 
can be written in terms of variables pi = -Px^,x^|x(0>0|0), p2 = Px„x,\xi^^ ^l^)' P^ = ^x^x^lxC^^^l^) 
and p4 = P^ ^ |x(^' ^1*-*)' ^4 ~ ^ — Pi — P2 — P3- However it is also easy to see that the distortion 
constraints are satisfied with equality, otherwise one can reduce the rate region slightly and still be under 
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Non-Causal Cribbing 


Strictly-Causal Cribbing 


No Cribbing 


Ro>{I{X;Xi,X2)-H{Xi)}+ 
(p.m.f.) : P{xJi,X2) 


Ro + Ri>I{X;Xi,X2) 
Ro > {I{X;Xi,X2) - H(Xi\X2)}+ 

(p.m.f.):P{X,X,,X2) 


Ro + Ri>I{X;X,,X2) 

Ro>I{X;X2) 

(p.m.f.) : P{X,Xi,X2) 



TABLE III 

Comparing rate regions for the example considered, for non-causal cribbing, strictly causal cribbing 

and no cribbing. 



distortion constraint. The distortion constraints thus yield, 

E[d{X,Xi)] = p^+p^ = Di 

E[d(X,l2)] = P2+PA = D2, 



(104) 
(105) 



which implies, p2 = 1 — ^i— Pi^Ps = 1 — ^2— Pi»P4 = P1 + D1 + D2 — I. Thus the equivalent probability 
distribution space over which the closure of rate regions is evaluated (such that distortion is satisfied) is 
equivalent to, V = {pi ^ [I - Di - D2, min{l - Di,l - D2, 2 - Di - D2}],P2 = I- Di-pi,p3 = 
I — D2 — pi,P4: = Pi + Di + D2 — I}. The various entropy and mutual information expressions appearing 
in the rate regions of non-causal, strictly causal and no cribbing (cf. Table III) can then be expressed as. 



I{X;Xi,X2) 
H{Xi) 

H{Xi\X2) 
I{X;X2) 



H2 

1 

Ho 



Pi +PA P2+ P3 P2 +P3 Pl+ PA 



H2{[P1 P2 P3 Pa\) 



Pi +PAP2+ P3 



l-Hn 



Pi +P3P2+ Pa 



(106) 
(107) 
(108) 
(109) 



where H2{-) stands for the binary entropy of the probability vector. Note the only variable of optimization 
is effectively pi. Fig. 13 shows the rate regions for {Di,D2) = (0.05,0.1). Note that the region for no 
cribbing is smaller than that of strictly causal cribbing which is smaller than that of non-causal cribbing. 
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Rate Regions 



Non-Causal Cribbing 
Strictly Causal Cribbing 
No Cribbing 




R (Common Rate) 



Fig. 13. Rate regions for non-causal, strictly causal and no cribbing in successive refinement setting of Fig. 4. Source is Bern(0.5) 
and (Di, -D2) = (0.05, 0.1). The curve is tradeoff curve between Ri and Rq and the rate regions lie to the right of the respective 
tradeoff curves. 



as expected. We can also analytically compute the expression of corner points A,B,C,D in Fig. 13. Let 
/12(a) = — aloga— (1 — a) log(l — a) V a G [0, 1]. Consider independent bernouUi random variables Z/)^ ~ 
Bem(Di) and Z£,2 ~ Bern{D2). Rq for point D is evaluated by putting i?i = in rate region for non-causal 
cribbing and this equals uiinp I{X; Xi, X2)- We will now show that rainp I{X;Xi,X2) = 1 — h2{Di). 
Consider, min-p I{X;Xi,X2) > min-p I{X;Xi) > minp(l - H{Xi\X)) > 1 - /i2(^i), where the last 
two inequalities follow respectively as Xi is Bern(0.5) and that Di is the hamming distortion between 
Xi and X. As D2 > Di, this lower bound is indeed achieved ii X2 = Xi = X (B Zd^. Similarly for 
point A, Ri is obtained by substituting i?o = in the expression of rate region for non-causal cribbing 
and this again equals 1 — h2{Di). Rq corresponding to points B and C is obtained by putting i?i = 00 in 
the expressions of rate regions of strictly-causal and no cribbing. Let us first consider point B and observe 
that Rq equals minp{/(X; Xi, X2) - H{Xi\X2)}^- We show that this equals 1 - /i2(-Di) - h2{D2). To 
see this, consider, minp{/(X; Xi,X2) - H{Xi\X2)}+ = minp{F(X2) - H{Xi,X2\X)}+ > minpjl - 



34 

H{Xi\X) — H{Xi\X)}^, where the last inequality follows as X2 is Bern(0.5). Since Xi and X2 are 
within hamming distortion Di and D2 to X respectively, we have m.mp{I{X; Xi, X2) — H{Xi\X2)}^ > 
1 - h2{Di) - h2{D2), where the equality holds for Xi = X ® Zd^ and X2 = X + Zo^- Similarly for 
point C, it can be shown Rq equals minp I{X] X2) = 1 — h2{D2). 

V. Dual Channel Coding Setting 

In this section we establish duality between cribbing decoders in the successive refinement problem and 
cribbing encoders in the MAC problem with a common message. The duality between rate-distortion and 
channel capacity was first mentioned by Shannon, [22] and was further developed for the case of side 
information by Pradhan et. al., [23] and by Chiang and Cover, [24]. Additional duality has been shown by 
Yu, [25] for a class of broadcast channels and multiterminal source coding problems, and by Shirazi et. 
al, [26] for the case of increased partial side information. The duality between source and channel coding 
with action dependent side information was shown in Kittichokechai et al. in [27]. Recently, Gupta and 
Verdii, [28] have shown operational duality between the codes of source coding and of channel coding 
with side information. 

To make the notion of duality clearer and sharper, we consider coordination problems in source coding 
[29] and for channel coding we consider a new kind of problems which we refer to as channel coding 
with restricted code distribution. In the (weak) coordination problem [29] the goal is to generate a joint 
typical distribution of the sources and the reconstruction (or actions) rather than a distortion constraint 
between the source and its reconstruction. Similarly, we define a channel coding problem where the code 
is restricted to a specific type. The achievability proofs for coordination and channel capacity with restricted 
code distribution are the same as that of rate-distortion and channel capacity, respectively, since the codes 
in all achievability proofs are generated randomly with specific distribution. The converse is also similar 
except in the last step where we need to justify the constraint of having a code with a specific type. For 
this purpose we invoke [29, Property 2] that is stated as follows : 

Lemma 5 (Equivalence of type and time-mixed variables [29]). For a collection of random sequences X^, 
y", and Z", the expected joint type EPx^,Y",Z'^ is equal to the joint distribution of the time-mixed variables 
{Xq, Yq, Zq), where Q is a r.v. uniformly distributed over the integers {1, 2, 3, ..., n} and independent of 

(X",y",z"). 
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The duality principle between source coding and channel coding with cribbing appears later in Table 
IV. According to those principles, the standard successive refinement source coding problem which 
was introduced in [18] is dual to the MAC with one common message and one private message [30]. 
Furthermore, the successive refinement source coding with cribbing decoders is dual to the MAC with one 
common message and one private message and cribbing encoders. To show the duality, let us investigate 
the capacity of the MAC with common message and cribbing encoders and compare it to the achievable 
region of the successive refinement problem with cribbing. 



A. MAC with cribbing encoders and a common message 

We consider here the problem of MAC with partial cribbing encoders where there is one private message 
nil G {1, 2, ..., 2"^i} known to Encoder 1 and one common message rriQ € {1, 2, ..., 2"^"} known to both 
encoders that needs to be sent to the decoder, as shown in Fig. 14 . We assume that Encoder 2 cribs 
the signal from Encoder 1 , namely. Encoder 2 observes a deterministic function of the output of Encoder 
1. We consider here three cases, noncausal, strictly-causal and causal cribbing and we show in the next 
subsection their duality to the successive refinement problem. 

Definition 4. A (2 ° , 2"^ , n, P{xi , X2)) partial cribbing MAC, with one private and one common message 
and a code restricted to a distribution P{xi,X2), has, 

1) Encoder 1, gi : {1, ..., 2"-^"} x {1, ..., 2"-^i} ^ A'f . 

2) Encoder 2, V i = 1, ..., n. (depending on d in Fig. 4, the decoder mapping changes as below), 

g^l : {!,..., 2"^"} xZf ^A's non-causal cribbing, d = n (110) 

g^.'^i : {!,..., 2"^"} x Z|"^ -^ X2 strictly-causal cribbing, d = i-l, (111) 

gli : {1,...,2"'^°} xZi-^ X2 causal cribbing, d = i. (112) 

3) Decoder, f : y" ^ {1, ..., 2"-^''} x {1, ..., 2"-^i}. 

An error occurs if the one of the messages was incorrectly decoded or if the joint type of the output 
and input to the channel deviates from the required one. Hence, the probability of error is defined for any 
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integer n and 5 > such as 

Pe(n),5 ^ Pr |(Mo(y«),Mi(y")) / (Mq, Afi) AND ||Pxr,x?,y"(x,y, z) - P{xuX2)P{y\xi,X2)\\TV > 'j} , 

(113) 
where Px^,x^,Y"i^^y^ ^) i^ '^'^^ joii^t type of the input and output of the channel and || • \\tv is the total 
variation between two probability mass functions, i.e., half the Li distance between them, given by 

\\p{x,y,z)-q{x,y,z)\\TV - i^'^\p{x,y,z) - q{x,y,z)\. 

x,y,z 

A pair rate {Rq, Ri) is achievable if for any 6 > there exists a sequence of codes such that Pe^^''^ — )■ 
as n — )• cx). The capacity region is defined in the standard way for MAC as in [31, Chapter 15.3], as 
the union of all achievable rate pairs. Let us define three regions 7^"'^, TZ^'^ and TZ'^, which correspond to 
noncausal, strictly-causal, and causal cases. 

Ro + Ri<I{Y;X,,X2), 

Ri < I(Y: Xi\X2,Zi) + H{Zi\X2) 
Rq + Ri<I{Y-Xi,X2). 

,, { Ri<I{Y-Xi\U,Zi) + H{Zi\U) 

PHxOi..^/(.,.„ I Ro + Ri<I{Y;Xi,U), 
where the union is over joint distributions that preserve the constraint P{xi, X2). Since X2 = f{u, zi), note 
that I{Y; Xi,U) = I{Y; Xi, X2). The next theorem states that the regions defined above, TV"'{P),n'''{P) 
and TZ^{P) are the respective capacity regions. 

Theorem 7 (MAC with common message and cribbing encoders). The capacity regions of MAC with 
common message, restricted code distribution P{xi,X2) and non-causal, strictly-causal and causal cribbing 
that is depicted in Fig. 14 are TZ"''^{P),TZ^'^{P) and TZ'^{P), respectively. 

The achievability and the converse proof of the theorem is presented in the Appendix. In the coding 
scheme of the achievability proof, we use block Markov coding, backward decoding and rate splitting 
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similar to the techniques used in Willems and Van der Muelen [1] and Permuter and Asnani [2]. The 
converse uses the standard Fano's inequaUties and the identification of an auxiUary random variable. 

B. Duality results between successive refinement and MAC with a common message 



SOURCE CODING 


CHANNEL CODING 


Source encoder 


Channel decoder 


Encoder input Xi 


Decoder input Yi 


Encoder output 

Mg{1,2,..,2"^} 


Decoder output 

Mg {l,2,..,2"-f^} 


Encoder function 

/:A'"^{1,2,...,2"^} 


Decoder function 

/: A'"^{1,2,..,2"^} 


Source decoder input 

Mg{1,2,..,2"«} 


Channel encoder input 

Mg {l,2,..,2"-f^} 


Decoder output X" 


Encoder output X" 


Cribbing decoders Zi{Xi) 


Cribbing encoders Zi{Xi) 


Noncausal cribbing decoder 

fi :{1,2,...,2"^} X Z"^li 


Noncausal cribbing encoder 

/, :{l,2,...,2"«}xZ"^Xi 


Strictly-causal cribbing decoder 

fi :{1,2,...,2"-^} X P^^^Xi 


Strictly-causal cribbing encoder 

/, :{l,2,...,2"«}xZ^-i^X, 


Causal cribbing decoder 

/i:{l,2,...,2"^}xZ^^l, 


Causal cribbing encoder 

/, :{l,2,...,2"«}xZ^^X, 


Auxiliary r.v. U 


AuxiUary r.v. U 


Constraint 

P{x,xi,X2), P{x) is fixed 


Constraint 

P{y, xi,X2), P{y\xi,X2) is fixed 


Joint distribution P{x,xi,X2,u) 


Joint distribution P{y,xi,X2,u) 



TABLE IV 
Principles of duality between source coding and channel coding 



Now that we have the capacity regions of the MAC with common message and of successive refinement 
we explore the duality of the regions. From a first glance at the regions of MAC with common message 
and of successive refinement, their duality may go unnoticed. However, the corner points of the regions 
are dual according to the principles presented in Table IV and as seen in Fig. 14. 

Tables VII- VI presents the corner points of the capacity region of the MAC with partial cribbing and 
common message and compare them to the corner points of the successive refinement (SR) rate region 
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MAC WITH COMMON MESSAGE 
AND CRIBBING ENCODERS 



SUCCESSIVE REFINEMENT 
WITH CRIBBING DECODERS 



Ml 



Xu(A'k,A'h) 



ENCODER 1 



M„ 



Ti = 3{X,] 



Pi 



Y\Xi,X^ 



ENCODER 2 



^2,,(Mo,2i") 









X" 



ENCODER 



Ti 



DECODER 1 



Xi"(To,Ti) 



DECODER 2 



Fi = g(^i) J 




y I(Y:Xi,X2) R„ 

{I(Y;X2.Zi)-H(Zi)}+ 




I(X;Xi,X2) /?„ 
{I(X;X2,Zi)-H(Zi)}+ 



Fig. 14. Duality between the cribbing decoders in successive refinement problem and the cribbing encoders in the MAC problem 
with a common message, non-causal case. Table IV represents how the expression of rate and capacity regions of the two problems 
are related. In the figure, for a fixed joint probability distribution, we plot the rate and capacity regions, and we observe that the 
comer points are dual to each other. Point Y corresponds to {Ro,Ri) — (0, /(X; Xi, X2) — {IiX',^2,Zi) — H{Zi)}'^) and 
Point X corresponds to {Ro,Ri) = {0, 1{Y; Xi, X2) - {I{Y;X2,Zi) - H{Zi)} + ). 





Corner points (Rq, Ri) of the noncausal (d = n) 


MAC 
Eq. (114) 


{I{Y;XuX2),0) 
{{I{Y; X2, Zi) - H{Zi)}+,IiY; Xi, X2) - {I{Y; X2, Zi) - H{Zi)}+) 


SR 
Theorem 2 


{I{X;X,,X2),0) 
{{I{X; X2, Zi) - H{Zi)}+, I{X- Xi, I2) - {I{X; I2, ^1) - H{Zi)}+) 



TABLE V 
The corner points of the noncausal case. 



with partial cribbing encoders. Note that applying the dual rules Xi -H- Xi, X2 -H- X2, Y ■H- X, and 
>-H'<, we obtain duality between the corner points of the capacity region of MAC with common message 
and the rate region of the successive refinement setting. 
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Comer points {R, Ri) of the strictly causal case (d = i — I) 


MAC 

Eq. (115) 


(/(F;Xi,X2),0), 
{{liY; X2, Zi) - HiZ,\X2)}+, I{Y- Xi,U) - {/(F; X2, Z^) - H{Z^\X2)y) 


SR 
Theorem 4 


{I{X;Xi,X2,0) 
{{IiX;X2,Zi) - H{Z,\X2)}^J{X;Xi,X2) - {IiX;X2,Zi) - HiZi\X2)}+) 



TABLE VI 

The corner fonts of the strictly causal case. 





Corner points {R, Ri) of the causal case (d = i) 


MAC 
Eq. (116) 


{I{Y; Xi, U), 0), where X2 = /{U, Zi) 
i{I{Y; U, Zi) - H{Zi\U)}+J{Y; Xi,U) - {/(F; U, Z^) - H{Z^\U)}+) 


SR 
Theorem 6 


{I{X- Xi, [/, 0) where X2 = f{U, Zi) 
{{I{X; U, Zi) - HiZi\U)}+, I{X; Xi,U) - {I{X; U, Zi) - H{Zi\U)}+) 



TABLE VII 

The corner foints of the causal case. 



C. Duality between MAC with conferencing encoders and successive refinement with conferencing decoders 

In the previous subsection we saw that there is a duality between the problem of MAC with one common 
message and one private message with cribbing encoders to successive refinement with cribbing decoders. 
Now we show that the duality also exists if the cooperation between the encoders/decoders is through a 
limited rate (conferencing) link as shown in Fig. 15. 

Theorem 8. The capacity region of MAC with one common message at rate Rq known to both encoders, 
one private message at rate Ri known to Encoder 1, and a limited rate link from Encoder 1 to Encoder 
2 at rate R12 with a restricted code distribution P{xi,X2) is 



Ro + Ri < IiXi,X2;Y) 

Ri < I{Xi;Y\X2) + R12. 



(117) 



This theorem can be proved using the result of conferencing MAC [32] where C21 = 00, and choosing 
U = X2. It is also possible to prove the theorem directly. The achievability part of Theorem 8 follows 
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easily if i?i < R12, then the conferencing Unk can be used to convey message Mi, thus both the encoders 
have a common knowledge of both the messages, so Rq + Ri < I{Xi,X2;Y) is achievable. If rate 
R12 < Ri, then the conferencing can be used to increase the common message rate to Rq + R12 and 
decrease the private message rate to Ri — R12. The converse can be proved using the fact that nRi = 
H{Mi) = H{Mi\Mq) < H{Mi2\Mo) + H{Mi\Mo, M12), and then bounding H{Mi2\Mo) < nRu using 
the fact that the cardinaUty of M12 is 2"^!^ and bounding iJ(Mi|Mo, M12) < nI{XiQ;YQ\X2Q) using 
Fano's inequality and the fact that the channel is memoryless. 

Finally, one can note a duality between the MAC with one common message and one private message 
and conferencing encoders to the successive refinement with conferencing decoders. In particular. Table 
VIII presents the corner points of the achievability regions of the two problems from which the duality 
rules Xi -ir^ Xi, X2 -^ X2, Y -ir^ X, emerge. 



MAC WITH COMMON MESSAGE 
AND CONFERENCING ENCODERS 



M, 



^i"(Mo,Mi; 



ENCODER I 



M„ 



Mi2(Afo,Afi) 



ENCODER 2 



F|Xi,X2 



XSiMo,Mu) 



Ri 



ii{I(Xi, X2; Y), I(Y; Xi 1X2) + R12} 



DECODER 



Wo(y ") 



Ri 



min{I(Xi, X2; X), I(X; X^|X2) + R12} 




7- 
{I(Y;X2)-Ri2}+ 



I(Y;Xi.X2 



-Ro 



Xn 



SUCCESSUVE REFINEMENT 
WITH CONFERENCING DECODERS 



ENCODER 



^1 r ^^"(^0,^1) 

DECODER I 



Tu{To,Ti) 



DECODER 2 



)X^{To,Tu) 



7 
{I(X:X2)-Ri2}+ 




I(X:Xi,X2) 



Fig. 15. Duality between the conferencing decoders in successive refinement problem and the conferencing encoders in the MAC 
problem with a common message, non-causal case. In the figure, for a fixed joint probability distribution, we plot the rate and 
capacity regions, and we observe that the corner points are dual to each other. 
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Comer points {Ro,Ri) of the conferencing case 


MAC 

Theorem 8 


{I{Y;X,, X2),0) 
(min(I(y; Xi, X2), /(F; XijXs) + R12), {HY; X2) - Ri2}+) 


SR 

Eq. (l)-(2) 


{IiX;Xi,Xi),0) 
(min(/(X; li, I2), HX; I1II2) + R12), {I{X; X2) - Ru}^) 



TABLE VIII 

The corner points of the achievabilities of the MAC with one common message and one private message 

AND conferencing ENCODERS AND OF SUCCESSIVE REFINEMENT WITH CONFERENCING DECODERS. 



VI. CONCLUSION 

In this paper, we introduced new models of cooperation in multi terminal source coding. The setting of 
successive refinement with single encoder and two decoders was generalized to incorporate cooperation 
between the users via (a) conferencing, or (b) cribbing. A new scheme,"Forward Encoding" and "Block 
Markov Decoding" was used to derive the rate regions for strictly-causal and causal cribbing. Certain 
numerical examples are presented and show how cooperation via cribbing can boost the rate region. 
Finally, we introduce dual channel coding problems, and establish duality between successive refinement 
with cribbing decoders and communication over the MAC with common message and cribbing encoders. 
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Appendix A 
Successive refinement with Conferencing Decoders, Fig. 3 

Consider Fig. 3, here Decoder 1 cooperates with Decoder 2 by providing an additional description T12 
to it. The rate region is given by, 

^0 + ^1 > I{X;Xi,X2) (118) 

R0 + R12 > I{X;X2), (119) 

for some joint probability distribution P^ ^ ^ such that E[di{Xi, Xi)] < Di, for i = 1,2. We will briefly 
describe the proof as they are based on standard arguments used throughout the paper. 
Achievability : We provide the achievability under two cases, 

• Case 1 : Ri < R12, here we describe Ti through T12, thus both the decoders know {Tq,Ti) and 
hence the following region is achievable, 

Ri < R12 (120) 

Ro + Ri > I{X;Xi,X2), (121) 
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for a joint probability distribution P^ ^ ^ such that distortion constraints are satisfied. The region 
is equivalent to, (call it Region 1) 

Ri < Ri2 (122) 

Ro + Ri > I{X;Xi,X2) (123) 

R0 + R12 > I{X;X2). (124) 

• Case 2 : Ri > R12, here Ti is described as a tuple {T{,T[') of rate (i?i — Ri2,Ri2), and T[' is 
described via the conferencing link. Thus this problem is similar to original successive refinement 
problem, where encoder has a private rate Ri — R12 and a common rate Rq + R12, and hence the 
following region (call Region 2) is achievable (follows from the achievability of Equitz and Cover 
[18]), 

Ri > R12 (125) 

Ro + Ri > IiX;Xi,X2) (126) 

R0 + R12 > I{X;X2), (127) 

for a joint probability distribution P^ j^ j^ such that distortion constraints are satisfied. 
We finish the proof of achievability by combining Region 1 and Region 2. Converse follows from standard 
cutset bound arguments and is omitted. 

Appendix B 
Proof of Lemma 1 

The proof is done by proving set inclusions in two directions as done for Theorem 3 in [20]. First we 
prove, TZ{Di,D2) C TZcascade{Di, D2) . Suppose a pair {Ro,Ro + Ri) G TZ{Di,D2). This implies there 
exists a (2"^" , 2"^^ , n) code (cf. Definition 2), for the setting of successive refinement with cribbing (Fig. 
4), such that distortion constraints Di + e and D2 + e we met at the decoders. We can use this code to 
generate a code of rates i?i = i?o + ^1 and R12 = Rq, for our cascade setting with cribbing (Fig. 5), with 
exactly same distortions at the decoders. This proves one direction. 

For the other direction, i.e., TZcascade{D 1,02) C n{Di,D2), assume, (i?i2,i?i) G T^cascadeiD 1,02), 
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which means there exist codes with rates {R12, Ri) with decoders incurring distortions, Di + e and D2 + e. 
Assume the messages sent on first and second Unk in our cascade problem be Ti and T12 respectively. 
T12 is a function of Ti, and we have 

nRi2 > H{Ti2) (128) 

nRi > H{Ti) = H{Ti,Ti2) = H{Ti2) + H{Ti\Ti2). (129) 

Using this code, we now will construct a code for successive refinement setting with cribbing decoders. 
Specifically, we consider encoding in B blocks where each block is of length n. Denote by Ti2{i) and 
Ti (i) the messages which are transmitted in the cascade source coding setting in i^^ block. Note that the 
tuple {Ti2(l),--- ,Ti2{B)} can be communicated to both decoders in the successive refinement setting 
with vanishing probability of error, with a rate Rq = ^H{Ti2), with large number of blocks, and similarly 
the tuple (Ti(l), • • • , Ti{B)) can be communicated to Decoder 1 with rate (using Slepian Wolf Coding as 
(Ti2{i),Ti{i)) are independent) Ri = -H{Ti\Ti2)- Thus Decoder 1 and Decoder 2 will know exactly the 
same (Ti2,Ti) and T12 respectively as they would know in cascade setting. Since the cribbing structure 
(Decoder 2 gets the crib from Decoder 1 non-causally, strictly-causally and causally) is same in cascade 
source coding and successive refinement setting, decoders will be able to achieve same distortion levels, 
pi, 1)2). This implies, {^H {T12) , ^H {Ti\Ti2)) G ^Pi,Z?2) or (^if (T12), ^//(Ti)) G 'JZ{Di,D2), 
which implies by Eq. (128)-(129), that (i?o,^i) G n{Di,D2). 

Appendix C 
Proof of Achievability in Theorem 2 

We describe in detail the achievablility in Theorem 2. 

• Codebook Generation : Fix the distribution PxPz x \x^x \x z x ' ^ ^ ^ ^^'^'^ ^^^^ E[di{X, Xi)] < 
3^ and E[d2{X,X2)] < ^. Generate codebook C^^ consisting of 2"^(^''^^) X^irrih) codewords 
generated i.i.d ~ P^ , mh G [1 : 2^^^-^'^^']. For each m/j, generate a codebook C^ {rrih) consisting 
Qj: 2ni(X;Zi\X2) ^n codewords generated i.i.d. ~ P^ ,-^ . We then bin these generated Z" codewords 
for each nih, in 2"^" vertical bins, B{my), m^ G [1 : 2"^"] and index them accordingly with 
Z G [1 : 2"(^("^'^il^2)--Ro)] 2^n codewords can be indexed equivalently as the tuple {mh,m^,l). For 
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each Zi{mh,my,l) codeword, generate a codebook, Cj^ {mh,my,l) consisting of 2"'^^-^'-^^'^^'^^' 
X^{mh,my,l,k) codewords generated i.i.d. ~ Px,\z, X2' ^ e [1 : 2"^('^'^il^i'^^)]. Thus the 
generation of codebooks is similar to that in perfect cribbing, except here we generate one more 
layer, of Zi codewords. Also we bin Zf codewords instead of Xf. Here, rrih and tUv correspond to 
the row and column index of the "doubly-indexed" bin which contains Z" codeword and for each Zf 
codeword, a codebook of Xf codebook is generated. 

Encoding : Given source sequence X", encoder finds the index irih G [1 : 2"^("^'^^)] from codebook 
C^ such that (X", X2{'mh)) € 7^". The encoder then finds the index tuple {m^^l) from the C^ {mh) 
codebook, such that {X"', Z^{mh,my,l),X2{mh)) G 7^". Encoder then finds the index k from the 
Cx {^TT'h,'mvJ) codebook, such that {X^,Xf{mh,m^,l,k),Zl'^{mh,mv,l),X2{mh)) G 7^". Thus 
Z" G B{my). rriy is described as Rq and the index triple, {mh, /, k) is described as Ri, thus 

Ri > I{X;X2) + I{X;Zi\X2)-Ro + IiX;Xi\Zi,X2) 
or, Ro + Ri > I{X-Zi,Xi,X2) = I{X;Xi,X2), (130) 

as Zi = g{Xi). 

Decoding : Using the indices sent by encoder. Decoder 1 constructs X" = X^{mh,my,l, k). Decoder 

2 gets Zi and column index m^, and infers the unique index nih such that Z^ = Z]^{mh, m^, I) for 

some l£ [I : 2"W^'^il^^)-^'')]. 

Distortion Analysis : Consider the following events : 



£0 = Encoder cannot find (X2', Z",Xf) jointly typical with given source X"- (131) 

But the probability of this event vanishes by Covering Lemma, Lemma 3 as there are 2'"'^^'^'^'^' 
X2 codewords, for each X2 codeword there are 2"-^^^'^^''^'^' Z" codewords and finally for each 
Zf codeword there are 2"^("'^'^il'^i'^2) xn codewords . Without loss of generality, now suppose 
that {nih, niy, I, k) = (1, 1, 1, 1) was sent by the encoder. 
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£i = Z'l does not lie in bin with row index vnh = 1 and column index rriv = 1 (132) 
= jZi" / Zi"(l, 1, [), for any [ G [1 : 2<^i.x-MX2)-Ro)A^ (133) 

But the probability of this event goes to zero, because of our encoding procedure, as Zf = 

Zf (1,1,1). 

£2 = Zi lies in bin with row index m/j / 1 and column index rriy = 1. (134) 

Zi" = Zi"(m/„ 1,1), nih / 1, for some [ G [1 : 2"(^(^'^il^^)-«°)]|. 

(135) 

Using similar argument as in the case of perfect cribbing, probability of this event goes to zero 
with large n, if 

I{X-Xi,X2)-Rq < I{Zi-Zi,X2) = H{Zi). (136) 

Thus consider the event, £ = SqVJ SiVJ £2, using Eq. (130) and Eq. (136), probability of this event 
goes to zero with large n if, 

Ro + Ri > I{X;Xi,X2) (137) 

Ro > {I{X-Zi,X2)-H{Zi)}+. (138) 

Distortion is bounded as in other sections. 

Appendix D 
Proof of Theorem 7, MAC with cribbing encoders and common message 

Proof of achievability of Theorem 7, noncausal case: The main idea of the achievability proof is to 
split message mi into two parts m'l and m'[ with rates i?'^ and R![ respectively, such that Ri = R[ + R'(. 
Message m[ is transmitted to Encoder 1 through the cribbing signal Z", while m'( remains as a private 
message to Encoder 1. 
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Code design: For the given joint distribution P{xi,X2) generate 2"^'i codewords zf distributed i.i.d. 
according to P{zi). For each codeword zf generate 2"^" codewords X2 according to P{x2\zi). For each 
codewords pair {z^^x"^) generate 2"^i' x" codewords according to P(xi|zi, 2:2). 

Encoding and decoding: 

• Encoder 1: maps {m'i,m'(,mo) to {zi{m'i),X2{zi,mQ),Xi{x2, zf,m'-l)), and transmits 
xUx'^,zl^,m'l). 

• Encoder 2: transmits X2{zi,mQ). 

• Decoder: looks for {mo,m[,m'() such that 

(zrK),4(^r>o),:r^K,^r>'i'),y") G ri"^- (139) 

Error analysis: Without loss of generality let's assume that the message that is sent is mo = 1, m'^ = 1, 
and m'( = 1. 

• Let Eo be the event that (x"(l), Xg (1), y") ^ T^ ■ Clearly, Pr{£'o} — > by the law of large numbers. 
Hence, for the rest of the events we can assume that (x"(l),X2(l)) G T^ . 

• Let El J be the event that 2;"(1) = Zi{j). And let Ei be the event that there exists an j / 1 such 
that zf (1) = z'i{j). Following from the definition Ei = Uj>i£'ij. Let's bound the probability of Ei 
using the union bound and the fact that Pr{Sfej} < 2~"('f^(^i)~^). 

Pr{^i} = Prm>i^ij} (140) 

< Y.^T{Ei,j] (141) 

< ^2-"(^(^i)-^) (142) 

i>2 

^ 2"(^i-'f^(^i)+^), (143) 



hence, if 



PrjE"!} — ^ as n — > 00. 



i?; < H{Zi), (144) 
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Let Eij^k be the event probability that for m[ = i, rho = j, and 7fi'( = k 

(zrK),x^(zr>o),x?(x^,zr>'/),2/") G tW. (145) 

Let Es be the event that exists an {i,j, k) / (1, 1, 1) such that -E'j,j,fc occurs. 

Pt{Es}<Ft{ \J i?„-fc}+Pr{ J ^,,,-fe}+Pr{ J i^i,,-,fc}. (146) 

«>2,i>l,fc>l i=lj>2,fe>l i=l,j=l,fc>2 

Now let's bound each term. Consider the first term in the RHS of (146) 

2""i,2"-"o,2""i' 

M U ^^.^■.^} ^ E 2-"(^(^-^-^^^^)-) 

«>2,j>l,fe>l i=2,j=l,k=l 

hence if 

Ro + Ri<I{Xi,X2;Y), (148) 

then the probability above goes to zero. Consider the second term in the RHS of (146) 
Pr{ \J E.^^^k} < Y. 2-"(^(^-^-^l^^)-^) 

i=l,i>2,A:>l j=2,k=l 

< 2niR„+R'^-I(Xi,X2;Y\Zi)+e) ^-^^^g^ 

hence if 

Ro + R'l<I{Xi,X2;Y\Zi), (150) 

then the probability above goes to zero. 
Consider the third term in the RHS of (146) 

Pr{ U ^^.J>} ^ 2"(^i'-^(^-^l^-^^)+^), (151) 

i=l,j=l,k>2 

hence if 

R'l<I{Xr,Y\Zi,X2), (152) 

then the probability above goes to zero. 
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Gathering (144), (148), (150) and (152) we obtain 



i?'i < H{Zi) (153) 

Ro + Ri < I{Xi,X2;Y) (154) 

Ro + R'l < IiXi,X2;Y\Zi) (155) 

R'l < I{Xi;Y\Zi,X2). (156) 

Using Fourier— Motzkin elimination [33] we obtain 

Ro + Ri < IiXi,X2;Y) (157) 

Ro + Ri < I{Xi,X2;Y\Zi) + HiZi) (158) 

Ri < I{Xi-Y\Zi,X2) + H{Zi). (159) 

Since I{Xi,X2;Y) < I{Xi,X2;Y\Zi) + H{Zi) the second inequality in (187) is redundant and 
therefore the region 

Ro + Ri < I{Xi,X2;Y) 

Ri < I{Xi;Y\Zi,X2) + H{Zi). (160) 

is achievable. ■ 



Proof of converse for the non causal case: Let (2"^" , 2"^^ , n) be a non causal cribbing MAC code as 

in) 

defined in Def . 4 with a probability of error Pe . Consider, 

R^ + Ri = H{Mo,Mi) (161) 

= /(Mo,Mi;y") + i7(Mo,Mi|y") (162) 

< /(Xi",X2";y") + ne„ (163) 

< Y.I{X^,^,X2,^;Yi) + nen (164) 

< nI{Xi,Q,X2,Q;YQ\Q)+nen (165) 

< nI{Xi^Q,X2,Q;YQ) + nen, (166) 



= 


H{Mi) 






= 


H{Mi\Mo) 






(a) 
< 


/(Mi;y"|Mo) 


+ 


ne. 
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where (a) follows from Fano's inequality where e„ = {- + Rq + RijPe , step (b) follows from the 
memoryless nature of the MAC and (c) follows from denoting Q as uniform random variable over the 
alphabet {1, 2, ..., n}. Now consider 

Ri = H{Mi) (167) 

(168) 
(169) 

< I{X^,Z^;Y^\X!^) + nen (170) 

= /(Zf; y"|X2") + /(Xi"; y"|X^\ Zf) + ne„ (171) 

(6) " 

< J]i7(Zi,,) + /(Xi,,;yi|X2,„Zi,,) + ne„ (172) 

i=l 

(c) JL 

< Y,H{Zi,Q\Q)+I{Xi,Q;YQ\X2,Q,Zi,Q,Q)+nen (173) 

i=l 

n 

< J]/7(Zi,Q) + /(Xi,Q;yQ|X2,Q,Zi,Q) + ne„, (174) 
j=i 

where the justification for (a), (b) and (c) follows from similar arguments as steps (a), (b) and (c) for 

bounding Rq + Ri. Since the rate pair is achievable, the code type is arbitrary close to the restricted 

distribution P{xi,X2) and using Lemma 5 we conclude that the distribution of Xi^q, X2,q is arbitrary 

close to the restricted distribution P{xi,X2)- Finally, by denoting Zi = Zq, Xi = Xi^q, X2 = -^2,Q and 

Y = Yq and taking into account that Pg is going to zero as n — )■ 00 we obtain that the region TZ^'^{P) 

upper bound the capacity region. ■ 

Proof of achievability of Theorem 7, strictly causal case: The main idea of the achievability proof is 
to combine the rate splitting idea that we used in the noncausal case with the Markov block coding. We 
assume that the transmission is done in a block of size nB where B is the number of subblocks and each 
subblock is of length n. Let mo,6, wii,b be the messages sent in block h. Similarly to the noncausal case, 
split message ttii 5 into two parts m'^ ^ and m'l j, with rates R'^ and R!{ respectively, such that Ri = R[+R'(. 
Message m'^ j, is transmitted to Encoder 1 through the cribbing signal, while m'( ^ remains as a private 
message to Encoder 1 . Because of the causality, the the message m'^ j, is known to Encoder 2 only at the 
end of block b. 
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Code design: For fixed a joint distribution P{xi,X2) generate 2"(^"+^'i) codewords Xg each associated 
with the pair of messages (?72o,f,,m'^ ^ ^). For each codeword X2 generate 2"^i codewords zf according 
to conditional distribution P{zi\x2) associated with m'^ j. For each codeword pair (zf ,^2) generate 2"^i 
codewords x" according to conditional distribution P{xi\zi,X2) associated with m'/^. 

Encoding and decoding: 

• Encoder 1: In block 6 maps (m'^ b~i^''^'i fe'"^! bj'^o.b) to (x2(w,o,fe,m'^ ^_-|^), zf (m'^ {,, X2),x"(m'/^,X2, ^f)), 
and transmits x"(m'j'^,a:2, -z")). 

• Encoder 2: Transmits ^2 (mo,^, ^1 ^-i)- Message m'^ i^_^ is known to Encoder 2 since at the end of 
block 6—1, z"(m'^ jj_^, a;2) and ^2 are known. 

• Decoder: Does backward decoding. We assume that when decoding block b message m'^ ^ is known 
and it looks for tuple (mo,b, rh'^ b-i^''^i fe) ^^'^^ ^^^^ 

(x5(mo,6>;,,_i),zrK^„x5),x;X<fe,x^,zr),y") e ^i"^- (175) 

£rror analysis: Without loss of generality let's assume that the message that is sent is mo,fe = 1, m[ j^ = 

l'K,6-l = 1' and K',6 = 1- 

• Let Eq be the event that (x"(l), X2(l)) ^ T^ ■ Clearly, Pr{£^o} — > by the law of large numbers. 
Hence, for the rest of the events we can assume that (x"(l),X2(l)) G T^ ■ 

• Let El be the event that in block 6—1 there exists an j / 1, such that z"(l) = ^"(i) for some 
codeword Xg. Similar to the analysis for the noncausal case 

Pr{^i} = 2"(-^i-^(^il^^)+^\ (176) 

hence, if 

R[ < H{Zi\X2), {111) 

Pr{i?i} — > as n — > 00. 

• Let Ei^j^k be the event probability that for -m'^ j_^ = i,mo,fe = j, and m'/^ = k, given that m!^ ^ is 
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known correctly from pervious subblock decoding: 

(x^(mo,6>'i,6-i),^rK,6,^2),^iK6,3;^,^r),y") e ri"^- (178) 

Let Es be the event that exists an {i,j, k) / (1, 1, 1) such that -E'j,j,fc occurs. 

Pr{^3}<Pr{ U Ei^j,k} + M \J E,,,-fe}. (179) 

(MV(l,l),fe>l ii,j)={l,l),k>2 

Now let's bound each term. Consider the first term in the RHS of (179) 

M U ^i^^^k} < 2"(««+^i-^(^^'^^'^^'^)+^), (180) 

(jjV(i,i),fc>i 

hence if 

Ro + Ri<I{X^,X2;Y) (181) 

then the probability above goes to zero. Consider the second term in the RHS of (179) 

Pr{ IJ Ei^,^k} < 2"(«"-^(^^'^l^^'^^)+^), (182) 

(Jj) = (l,l),A:>2 

hence if 

R';<I{Xr,Y\Zi,X2), (183) 

then the probability above goes to zero. 
Gathering (177), (181), and (183) we obtain 

R[ < H{Zi\X2) (184) 

R^ + Ri < I{Xi,X2;Y) (185) 

R'l < I{Xi;Y\Zi,X2). (186) 

Using Fourier— Motzkin elimination 

Ro + Ri < I{XuX2;Y) (187) 

Ri < I{Xi;Y\Zi,X2) + H{Zi\X2). (188) 
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is achievable. ■ 

Proof of converse for the strictly causal case: Let (2"^i , 2"^", n) be a strictly causal cribbing MAC code 

(n) 

as defined in Def . 4 with a probability of error Pg ■ Following the exact same steps as in the converse of 
the noncausal case in (189) we obtain 

Rq + Ri < nI{Xi^Q,X2,Q;YQ) + nen. (189) 

Following the exact same first four steps as in converse of the non causal case to bound Ri, (167) we 
obtain 

Ri < I{Z'^;Y''\X^) + I{X'^;Y''\X^,Z^) + nen (190) 

n 

< Y,H{Zl,,\X2,^)+I{X,^^■,Y^\X2,^,Zl,i)+nen (191) 

4 = 1 

n 

< ^H{Zi,Q\X2,Q) + I{Xi,Q;YQ\X2,Q,Zi^Q) + nen, (192) 
1=1 

Since the rate pair is achievable, the code type is arbitrary close to the restricted distribution P{xi,X2) 

and using Lemma 5 we conclude that the distribution of Xi^q, X2,q is arbitrary close to the restricted 

distribution P{xi,X2)- Finally, by denoting Zi = Zq, Xi = Xi^q, X2 = ^2,Q and Y = Yq and taking 

into account that Pe is going to zero as n — )■ cx) we obtain that the region 7^"'^(P) upper bound the 

capacity region. ■ 

Proof of achievability of Theorem 7, causal case: In this proof we show how the causal case achievability 
follows directly from the proof of the strictly causal case with one modification: instead of codewords X2 
we generate codewords n", and the input to the channel is X2,i = f{ui,xi^i). This is possible since Encoder 
2 observes causally the signal from Encoder 1. By replacing X2 with U in 1Z^'^{P) and applying X2^i = 
f{ui, zi^i) and taking into account the equality I{Y; Xi, U) = I{{Y] Xi, U, f{Zi, U)) = I{Y; Xi, X2) we 
obtain the region 1Z^{P) . ■ 

Proof of converse for the causal case: Let (2"^" , 2"^^ , n) be a partial strictly causal cribbing MAC code 

(n) 

as defined in Def. 4 with a probability of error Pe . Following the exact same steps as in the converse of 
the noncausal case in (189) we obtain 

Rq + Ri < nI{Xi^Q,X2,Q;YQ) + nen, (193) 
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Now consider 

Ri = H{Mi) (194) 

= H{Mi\Mq) (195) 

(a) 

< /(Mi;y"|Mo) + ne„ (196) 

< /(Xr,Zr;y"|Mo) + ne„ (197) 
= /(Zi";y"|Mo) + /(Xi";y"|Mo,Zi") + ne„ (198) 

(6) " 

< J]/(Zi,,;y"|Mo,Z*-i) + /(Xi,i;y|X2,„Mo,Zi",X^-i) + ne„ (199) 

i=l 

(c) " 

< J]F(Zi,i|Mo,Z*-i) + /(Xi,,;y|X2,„Mo,Z^i)+ne„ (200) 

i=l 

< J]F(Zi,Q|[/Q) + /(Xi,Q;yQ|X2,Q,^Q)+ne„, (201) 

where (a) follows from Fano's inequality where e„ = (^ + i?i)Pe , step (b) follows from the memoryless 
of the MAC and (c) follows from denoting Ui = (Mq, Z't^^) and Q as uniform random variable over the 
alphabet {1, 2, ..., n}. Note that indeed X2,i = /{Mq, Z\^^,Xi^i) and therefore X2,q = /([/q, X2,q). Rest 
of the steps for the completion of proof follow similar arguments as in non causal and strictly causal case. 



